Touchpad-enabled remote controller and user interaction methods

ABSTRACT

The hand held case of the remote control unit includes at least one touchpad, and other sensors, such as acceleration sensors, case perimeter sensors, pressure sensors, RF signal sensors. These sensors provide a rich array of sensory inputs that are classified by a pattern recognizer to generate control commands for both the consumer electronic equipment and the remote control unit itself. A power management system to conserve unit battery power is also responsive to the pattern recognizer to allow intelligent power management control. The control system uses the display of the consumer electronic equipment to provide instructions to the user, and the behavior of the remote control system uses what is displayed on the display as context information for pattern recognition.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Patent Applications No.60/946,804 and Ser. No. 11/977,348 filed on Jun. 28, 2007 and Oct. 24,2007, respectively. The disclosures of the above applications areincorporated herein by reference.

BACKGROUND

Practically all consumer electronic products in use today come with aremote control. In most cases, the remote control has many buttons, eachdedicated to the control of one or more specific features of theconsumer electronics product. As these products increase in complexity,so does the number of buttons required. At some point, the increasednumber of buttons renders the remote control mostly useless for a largenumber of users. Various solutions have been proposed to address suchproblems. One solution is disclosed in U.S. Pat. No. 6,765,557 to use atouchpad for controlling a home entertainment device such as anInteractive television. Although the use of the touchpad prevents theincrease of buttons and thereby reduces the problem of userinconvenience, this is not enough to solve the problem completely.

The present invention takes a fresh look at the remote control ofconsumer electronic products. It replaces the large number of buttonswith a simple handheld remote control unit that includes at least onetouchpad together with a rich array of additional sensors, such asacceleration sensors, case perimeter sensors, pressure sensors, RFsignal sensors, and the like. The remote control system responds tosignals from this rich array of sensors using pattern recognitiontechnology that allows the system to control many complex functionswithin consumer electronic equipment based on the manipulation andgestural movement of the remote control by the user. The patternrecognition system is adaptive. It can identify different users by themanner in which the remote control system is utilized and it can adaptso that each user can manipulate the system in his or her own uniqueway, and still effect control over the various functions of the consumerelectronic equipment.

The remote control unit, itself, can also respond to the sensory input“meaning” to after the manner in which the remote control unit behaves.In this way, visual, audible or tactile cues, as well as powerconservation strategies, can be controlled and revised based on thesensory input data.

For a more complete understanding of the invention, refer to thefollowing description and to the accompanying drawings.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 a is a perspective view of the remote control unit.

FIG. 1 b is a plan view of the remote control unit.

FIG. 1 c is a view of the remote control unit in a portrait orientation.

FIG. 1 d is a view of the remote control unit in a landscapeorientation.

FIG. 2 is a system block diagram illustrating the remote control systemin operation by a user to control a piece of consumer electronicequipment.

FIG. 3 is a block diagram illustrating an exemplary embodiment of theremote control system, including components associated with the controlcircuit coupled to the consumer electronic equipment and associated withthe remote control unit.

DESCRIPTION OF PREFERRED EMBODIMENTS

Referring first to FIGS. 1 a and 1 b, the remote control unit 20 of theremote control system has been illustrated. This remote control unit 20interacts with a control circuit that is coupled to the consumerelectronic equipment. The control circuit and consumer electronicequipment have not been showed in FIGS. 1 a-1 d but are shown insubsequent FIGS. 2 and 3.

The remote control unit 20 has a touchpad 22 that may include predefinedclickable regions, such as the up-down-left-right-okay region 24, thechannel up-down region 26, the volume up-down region 28 and the muteregion 30. It will be understood that these predefined clickable regionsare merely exemplary of the basic concept that the touch screen can haveregions that respond to pressure as a way of signifying that the userhas “selected” a particular function. While the basic design of theremote control unit strives to eliminate physical push buttons to alarge extent, the remote control unit 20 may still have physical pushbuttons if desired. Thus, for illustration purposed, four push buttonsare shown at 32, 33, 34 and 35. It is also contemplated that thetouchpad 22 may be split into two distinct zones with or without aphysical divider interposed between the two zones.

The pre-defined clickable regions may be visually designated on thetouchpad surface by either silk screening the region graphics onto thesurface of the touchpad 22, or by using a see-through graphic withbacklighting. As will be more fully discussed below, the backlightingcan be triggered by the appropriate combination of sensory inputs asrecognized by the pattern recognizer also discussed below. It iscontemplated that the touchpad surface may not include any pre-definedclickable regions.

The case of the remote control unit 20 is preferably provided with aseries of capacitive sensors, such as sensors 36 around the horizontalside walls of the case perimeter. The capacitive sensors 36 are exampleof case perimeter sensors. Capacitive sensors 36 can also be at otherlocations, such as on the underside of the case. These sensors detecthow the user is holding the remote control. In this regard, differentusers may grip the remote control in different ways and the capacitivesensors 36 are arranged to be able to discriminate these different waysof holding the remote control. Although there may be subtle differencesin how one user holds the remote control as compared with another, thepattern recognition system, discussed below, can use this information torecognize these subtle differences. Moreover, the sensors in cooperationwith the pattern recognition system enable a user to operate the remoteindependently of how the remote is being held.

Referring now to FIG. 2, an overview of the pattern recognition systemwill be presented. FIG. 2 illustrates the remote control unit 20 beingmanipulated by a user 40 to operate a consumer electronic equipmentcomponent 48 having a display screen 50. The consumer electronicequipment 48 conventionally has its own electronics that are used toprovide the equipment with its normal functionality. In the case of theillustrated component 48 such functionality includes displaying audiovisual material on the display screen 50. This material may include, forexample, television programs, pre-recorded content, internet content andthe like. For illustration purposes, the associated electronics of theconsumer electronic equipment 48 have been illustrated separately at 52.Embedded within the electronics package 52 is a control circuit showndiagrammatically at 60 that defines part of the remote control system.Control circuit 60 is coupled to the consumer electronic equipment 48and responds to commands sent from the remote control unit 20 to controlthe operation of the consumer electronic equipment 48.

The remote control system is made up of the remote control 20 and thecontrol circuit 60. Together, these two components implement asophisticated sensory input detecting and pattern recognizing systemthat allows the user 40 to control operations of the consumer electronicequipment 48 using a rich variety of finger, hand, wrist, arm and bodymovements. The system may be viewed as effecting a dialogue between theremote control unit 20 and the control circuit 60, where that dialogueis expressed using a vocabulary and grammar associated with a diversevariety of different sensory inputs, (e.g., from the touchpad 22,accelerometer 114, case perimeter, sensor, pressure sensors, RF signalsensors and the like). The control system also includes a feedback loopthrough the user 40. The user 40 has his or her own set of user sensoryinputs 62 (sight, sound, touch) and the user 40 manipulates the remotecontrol unit 20 based, in part, on audible and visual informationobtained from the consumer electronic equipment 48, and on visual,audible and tactile information from the remote control unit 20. Thus,the remote control system supports a dialogue between remote controlunit 20 and control circuit 60, with a concurrent dialogue between user40, the control system and the consumer electronic equipment 48.

FIG. 2 thus illustrates that user 40 may receive visual, audible ortactile feedback from remote control 20 and this may be performedconcurrently while viewing the display screen 50. For illustrationpurposes, the information acquired by user 40 are depicteddiagrammatically as user sensory inputs 62. Likewise, the sensory inputsacquired by the control system (from a diverse array of different typesof sensors) has been diagrammatically illustrated at 64.

The relationship between the control system sensory inputs 64 and theuser sensory inputs 62 is a non-trivial one. The user 40 will manipulatethe remote control unit 20, in part, based on what the user 40 is tryingto accomplish and also, in part, based on what the user 40 sees on thedisplay screen 50 and what the user 40 also senses audibly, visually ortactilely from the remote control unit 20 and/or consumer electronicequipment 48. To illustrate this point, imagine that the consumerelectronic equipment 48 is a television set that has been programmed toblock certain channels from being viewed by young children. In order tobypass the parental blocking feature, user 40 must manipulate the remotecontrol unit 20 in a predefined way. To prevent the child from simplywatching the parent and learning the manipulating technique, theparental blocking unlocking feature can be changed each time it is used.The adult user must watch what is shown on the display screen 50 inorder to learn how to manipulate the control unit to unlock the parentalblocking feature. The instructions on the display are presented in aform, such as textual instructions, that a young child is not able toread. Thus, the control of the parental blocking feature relies on aparticular manipulation (e.g., flick the wrist three times) that iscontext-based. A later unlocking operation would be treated as adifferent context and would potentially have a different gesturalcommand to effect unlocking. Although this is but one example theexample illustrates that the behavior of the remote control system iscontext-dependent and that the user's sensory perception (e.g., readingthe screen, feeling tactile vibrations, hearing particular sounds) willaffect how the user's manipulations of the remote control unit 20 areinterpreted.

The control system is able to make sense of a rich and diversecollection of sensory inputs using a pattern recognizer 70 andassociated control logic 72. As the user 40 manipulates the remotecontrol unit 20, sensory inputs are collected as a temporal sequencefrom the various sensors within the remote control unit 20. Aspreviously noted, the sensors may include at least one touchpad 22responsive to manipulation by a user's fingers and at least oneadditional sensor such as, for example, an acceleration sensorresponsive to movement of the remote control unit 20, case perimetersensors such as capacitive sensors 36 that discriminate which parts ofthe case are in contact with the user's body, pressure sensorsresponsive to pressing forces upon a predetermined region of thetouchpad 22 and RF signal sensors responsive to radio frequency signalstransmitted from the control circuit 60.

The temporal sequence of sensory inputs is fed to the pattern recognizer70. The pattern recognizer 70 is configured to classify the receivedsensory input message according to a predetermined recognition scheme togenerate message meaning data that are then sent to the control logic72. The pattern recognizer 70 classifies a received sensory inputmessage based on context information extracted from knowledge of what isbeing displayed on the display screen 50 of the electronic equipment.The control logic 72 is responsive to message meaning data to generateat least one command that is communicated to the control circuit 60. Theoperations are described below in more detail. The control logic 72decodes the message meaning data and generates a device control signal.The device control signal may be supplied to the remote control unit 20itself, to effect control over the behavior of the remote control unit20 (e.g., putting the unit to sleep or waking the unit up) or the devicecontrol signal may be sent to and/or used by the control circuit 60,where it is passed on to the consumer electronic equipment 48 as acommand to control the operation of the consumer electronic equipment48. The pattern recognizer 70 and the control logic 72 may beimplemented separately or together and may be deployed in the controlcircuit 60, in the remote control 20, or distributed across both.

In one embodiment, the pattern recognizer 70 employs a trained modelthat may be adaptively altered or customized to more closely fit eachuser's style of using the remote control unit 20. The trained model isused to classify the received sensory input message. Here, the trainedmodel is adaptable based on interaction with the user 40. In suchtrained model embodiment, the pattern recognizer 70 is preferablyprovided with an initial set of models that classify certain operationsas being mapped onto certain commands or control functions. For example,with reference to FIG. 1 b, an upward sliding motion of the fingertip onchannel up-down region 26 might launch a forward channel scanning mode,whereas a single click or finger press upon the upward arrow of theregion 26 would simply increment the channel by one. This behavior mightbe classified differently, however, if the remote control unit 20 ispositioned in landscape orientation as illustrated in FIG. 1 d. Forexample, when in landscape orientation and held by two hands (asdetermined by the capacitive sensors 36), the channel up-down region 26might perform a function entirely unrelated to channel selection.

To adapt the model for a particular user, the preferred embodimentincludes a sensory input mechanism to allow the user 40 to inject a metacommand—to let the system know that the user 40 wishes to after thepattern recognition models either for himself or herself, or for allusers. For example, a rapid back and forth wrist motion (analogous toshaking one's head in a “no” gesture) might be used to inform therecognition system that the most recent pattern recognition conclusionwas wrong and that a different behavior is desired. For example, assumethat the user 40 has used the remote control unit 20 on a coffee tableand then manipulates the channel up-down region 26, causing thetelevision to begin a channel-scanning mode. Perhaps the user 40 wouldprefer that the channel scanning mode should not be initiated when theremote control unit 20 is resting on the coffee table (i.e., not beingheld). To change this behavior, the user 40 would pick up the remotecontrol unit 20 and shake it back and forth in a “no” gesture. Thiswould cause an on-screen prompt to appear on the television displayscreen 50, instructing the user 40 how the most recent temporal sequenceof sensory inputs can be modified in this context to result in adifferent device control signal outcome.

Because the pattern recognizer 70 can respond to a rich variety ofdifferent types of sensory inputs, the control system is able tointerpret the meaning of user 40 manipulations and gestures that can bequite complex, thereby allowing the user 40 to interact in an intuitiveor natural way that can be customized from user to user. In this regard,there may be instances where two or more gestural commands might be verysimilar and yet might have different meanings and thus might requiredifferent commands to be sent to the consumer electronic equipment 48.To handle this, the pattern recognizer 70 may be based on a statisticalmodel where the control system sensory inputs 64 generate probabilityscores associated with a plurality of different meanings. The patternrecognizer 70 would (a) select the meaning with the highest score, ifthat score is above a predetermined probability threshold value and/orabove the next-most value by a predetermined threshold, or (b) engagethe user 40 in a dialogue on-screen to resolve which meaning wasintended, if the preceding threshold conditions are not met. The resultsof such user interaction may then be used to fine tune or adapt themodel so that the system learns what behavior is expected for subsequentuse.

With the above overview in mind, refer now to FIG. 3 where a detaileddescription of the remote control unit 20 and control circuit hardwarehas been illustrated. In FIG. 3, the components associated with thecontrol circuit are shown generally at 60 and the components associatedwith the remote control unit are shown generally at 20. The consumerelectronic equipment is shown at 48.

Beginning with the control circuit 60, a first processor or CPU 80 isattached to a bus 82, to which random access memory 84 and programmablenonvolatile random access memory 86 are attached. The random accessmemory 84 and the programmable nonvolatile random access memory 86 arememories for storing control system operation information. The firstprocessor includes an input/output (I/O) module 88 that provides an I/Obus 90 to which an RF communication module 92 and consumer electronicproduct interface 94 are attached. The consumer electronic productinterface 94, in turn, couples to the remaining circuitry of theconsumer electronic equipment 48. The radio frequency communicationmodule 92 includes an antenna and is designed to communicate with acorresponding communication module associated with the remote controlunit 20. With the above structure, the control circuit 60 selectivelyprovides said operation information to the consumer electronic equipment48 for presentation to the user 40. Here, the control circuit 60provides the control system operation information to the consumerelectronic equipment 48 in accordance with said at least one commandgenerated by the control logic 72. The control system operationinformation may be integrated with different message meaning data. Inthis case, the control circuit 60 selectively provides the operationinformation to the consumer electronic equipment 48 in accordance withthe message meaning data generated by the pattern recognizer 70.

The remote control unit 20 has a second processor 96 with associated bus98, random access memory 99 and nonvolatile programmable random accessmemory 100. The processor 96 also has an I/O module 102 that supports anI/O bus 104 to which a variety of sensors and other devices may beattached. Attached to the I/O bus 104 is the RF communication module 106that communicates with its counterpart module 92 of the control circuit60. The display illumination device 108 is also coupled to the I/O bus104 so that the backlighting can be switched on and off to render anybacklight graphical elements on the touchpad 22 visible or invisible. Atactile feedback annunciator/speaker 110 is coupled to the I/O bus. Theannunciator/speaker 110 may be activated to produce tactile feedback(vibrations) as well as audible tones. The above-described operationsperformed by the pattern recognizer 70 and the control logic 72 arecontrolled by the second processor 96, the random access memory 99, thenonvolatile programmable random access memory 100, and the like includedin the remote control unit 20.

As previously discussed, the remote control unit 20 includes anassortment of different sensors. These include the touchpad 22 ortouchpads 22, a button pad membrane switch assembly 112, accelerometer114 and capacitive sensors 36. The button pad membrane switch assembly112 may be physically disposed beneath the touchpads 22 so that pressureupon the touchpad 22 will effect a switch state change from off to on.If desired, the button pad membrane switch assembly 112 may employpressure-sensitive switches that can register a range of pressures, asopposed to a simple on/off binary state.

Because the remote control unit 20 is designed to sit on the coffeetable when not in use, a battery power supply 200 is preferred. Thus,the power supply 200 includes a removable battery 202 as well as a powermanagement circuit 204. The power management circuit 204 supplies powerto the second processor 96 and to all of the modules within the remotecontrol unit 20 requiring power. Such modules include all of thesensors, display illumination 108, and annunciator/speaker 110 attachedto the I/O bus 104. If desired, an RFID tag 206 may be included in theremote control unit circuitry. The RFID tag 206 can be used to helplocate the remote control from the control circuit 60 in the event theremote control unit 20 is lost.

FURTHER IMPLEMENTATION DETAILS OF PREFERRED EMBODIMENTS The TouchpadSensor

The touchpad 22 is divided into plural separate regions of interaction,such that each region of interaction is interpreted to have a differentmeaning by the pattern recognizer 70. In more detail, the touchpad 22performs the following operations. The touchpad sensor can be segmentedto provide several different intuitive zones of interaction. Thetouchpad 22 is also clickable by virtue of the button pad membraneswitch assembly 112 located beneath or embedded within it. The clickabletouchpad 22 can register pressure information and react to pressure(both mechanically and electrically) by sending a specific signal whileproviding sufficient haptic feedback to the user 40 such as throughvibrations and sounds via the annunciator/speaker 110. The touchpad 22allows for the use of at least two contact points simultaneously. (e.g.,two finger input) such as one contact point per side of the pad. Thetouchpad 22 can be viewed as divided in two along a medial line (e.g.,separating the right and left sides of the touchpad 22 when held in alandscape orientation). The touchpad 22 can thus be constructed usingtwo single-position registering touchpads mounted side by side, or onesingle mufti-touch touchpad with the ability to register with equalprecision (two points of contact at the same time).

<Physical Buttons>

Although not required in all embodiments, the remote control unit 20 mayhave a complement of physical buttons. In this regard, four buttons32-35 have been illustrated in FIGS. 1 a and 1 b. These physical buttonsmay be implemented using the same button pad membrane switch assembly112 (FIG. 3) embedded beneath the touchpad 22. The physical buttons,like the context-dependent virtual buttons on the touchpad surface, canbe backlit to reveal button function names.

<Redefining Regions of Interaction>

To allow for natural operation, the remote control unit 20 uses itspattern recognition system to interpret the sensory data. The remotecontrol unit 20 further includes a case orientation sensor capable ofdifferentiating between portrait orientation and landscape orientation.The detection of the orientations of the remote control unit 20 may beperformed using the accelerometer 114, the capacitive sensors 36, andthe like, without the case orientation sensor. Included in the sensorydata are inputs from the accelerometer 114 or accelerometers 114 and thecapacitive sensors 36 placed around the periphery and the bottom of thecase. The user 40 will naturally turn the remote control unit 20 in hisor her hands to best accommodate what he or she is trying to accomplish.The pattern recognition system interprets how the user 40 is holding theremote control unit 20 and redefines these zones of interaction so thatthey will appear to be at the same place, no matter how the remote isoriented. For instance, the remote control unit 20 can be used with oneor two hands, and in both landscape and portrait orientation. This meansthat the pattern recognition system can detect: whether the user 40 isholding the remote control unit 20 by the right hand or by the lefthand; by which hand the user 40 is operating the touchpad 22; which partof the remote control unit 20 the user 40 is holding; and the like. Theregions of interaction are remapped onto different locations within thetouchpad 22 based on case orientation. The following describes moredetails. The pattern recognition system can discriminate the differenceand will automatically redefine the zones of interaction so that theuser 40 can perform the most probably operations in the easiest mannerfor that user 40. The zones of interaction include, for example,different zones within the touchpad 22. Different regions of thetouchpad 22 may be dedicated to different functions or different usermanipulation styles. In addition, the remote control unit 20 itself canbe manipulated into different virtual “zones of interaction” byemploying different gestures with the remote in mid-air, such as a quickflick of the wrist to change channels.

<Power Management>

The presently preferred embodiment is contemplated for very low powerconsumption. The remote control unit 20 includes power managementsystem. The control logic 72 generates at least one command that iscommunicated to the power management system to after power managementsystem behavior based on sensor input messages or said message meaningdata. The power management circuit 204, the power supply 200, and theremovable battery 202 shown in FIG. 3 form the power management system.For example, the remote control unit 20 may run on a single AA or AAAbattery or batteries for approximately one year. The power managementsystem performs: placing of the sensors included in the remote controlunit 20, into a sleep mode; detecting movement of the remote controlunit 20 by waking the acceleration sensor up in the sleep mode;determining whether or not the remote control unit 20 is held by theuser 40, by waking the case perimeter sensor up when the movement of theremote control unit 20 is detected; and releasing the sleep mode whenthe detecting detects that the remote control unit 20 is held by theuser 40. The power management system releases the sleep mode, by wakingthe touchpad 22 up and then waking said RF signal sensor up. Thefollowing describes these operations in more detail. With currentlyavailable technology, the wireless circuitry associated with RF modulesconsumes more power than the touch sensors; and the accelerometers 114and actuators consume less power than the touch sensors. For thisreason, the power management circuitry 204 places the wireless circuitryin the sleep mode (or turned off altogether) after a short period oftime after the remote control unit 20 is no longer being used (e.g., 30seconds). The touch sensors will then be placed in sleep mode (or turnedoff) after a somewhat longer period of time (e.g., 2 minutes). This willallow turning on the wireless circuitry again (in case the user 40touches the surface of the touchpad 22 or picks up the unit within twominutes). The accelerometers 114 are put into a low power mode where thecircuitry checks the accelerometer status at a much lower rate than thenormal accelerometer refresh rate. In this regard the normal refreshrate might be on the order of 50 Hz whereas in the low power mode therefresh rate might be in the order of 1 Hz, or even 0.1 Hz. The powermanagement circuitry 204 would implement a turn on sequence that isessentially the reverse of the turn off sequence, with the accelerometerrefresh rate being increased to full rate first, followed byreactivation of the touch sensors and finally by activation of thewireless circuitry. In the sleep mode, the RF modules may periodicallybe awakened, to check to see if there are any pending messages from thecontrol circuit 60.

In the presently preferred embodiment, the remote control unit 20 doesnot have a dedicated power-on button, as this might be a potentialsource of user confusion as to whether such button powers on the remotecontrol unit 20 or the television.

Thus, the pattern recognition system is used to handle power-on in anefficient manner. The remote control unit 20 turns on when the user 40first picks it up. For this reason, the system first checks the lowerresolution acceleration data to determine if the remote has been moved.If so, the capacitive sensors 36 are next energized to determine if theremote is actually being held (as opposed to simply being inadvertentlypushed or moved when resting on the coffee table). If the patternrecognition system determines that the remote control unit 20 is beingheld, then next the touchpads 22 and finally the wireless circuitry areactivated.

Alternatively, power-on can be triggered by a specific gesture, such asshaking the remote control unit 20. More complex power-on operation canalso be utilized, for example, to enforce parental control as discussedabove in connection with parental blocking features.

The pattern recognition system will likewise detect when it is time toturn the remote control unit 20 off by detecting inactivity, or ifdetecting that the television has been turned off. This latter eventwould be detectable, for example, by information communicated via the RFmodules.

<Remote Finder>

The control circuit 60, associated with the consumer electronicequipment 48, may include a button that will send a remote locationmessage to the remote control unit 20. More specifically, the controlcircuit 60 sends a find-me message that causes the remote control unit20 to perform an attention attracting operation useful in assisting theuser 40 to find the remote control unit 20 when it has become misplaced.The following describes more details. The user 40 would push this buttonif the remote control unit 20 has gotten misplaced. The control circuit60 would then periodically send a tell-me-where-you-are signal (thefind-me message) to the remote via RF. When the remote control unit's RFmodule next wakes up and finds the wake up signal, it will activate thehaptic feedback system (e.g., annunciator/speaker 110) causing the unitto make sound and/or vibrate and optionally use the display illuminationcircuitry 108 to turn the backlighting on. In addition, if desired, theremote control unit 20 and the control circuitry can use RF rangingfunctionality to measure the distance between the remote control unit 20and the control circuit 60. This information has been used to displaythe distance on the display screen 50, or even present a picture of theroom with highlighted areas identifying where the remote control unit 20could be. Alternatively, the RFID tag 206 may be used, allowing theprecise location of the remote control to be displayed on the displayscreen 50.

<Tight Coupling Between Remote Control System and On-Screen UserInterface>

As illustrated by the previously discussed example regarding parentalcontrol, the remote control system is able to capitalize on its tightcoupling with the on-screen information. The on-screen information, suchas instructions on how to deactivate the parental blocking feature, maybe stored in the programmable random access memory 86 of the controlcircuit 60 (FIG. 3) and may then be projected onto the display screen 50as an overlay upon the presently viewed program. First, by displayinginformation to the user 40 on the display screen 50, the user 40 doesnot need to look at the remote control unit 20 in order to operate it.If the user 40 needs to enter input, such as a spelled word, an overlayimage of a keyboard may be presented and the user 40 can navigate to thedesired keys by simply manipulating the touch screen while watching acursor or cursors (one for each finger) on the displayed overlaykeyboard. If desired, the remote control system circuitry can alsoobtain program guide information and the display overlay can then allowthe user 40 to select which programs to view or record by simplymanipulating the touch screen.

One can better understand the effectiveness of the remote control systemby considering where the functionality of the system has been placed. Bytight integration with the display screen 50, the remote control systemcan use the display screen 50, with its high resolution graphicscapability, to provide an unlimited amount of visual information to theuser 40 which would be virtually impossible to provide through a set ofdedicated buttons as conventional controllers do. The rich collection ofdiverse sensory inputs allows the user 40 to adopt many different, andeven redundant, ways of communicating the user's desires to the system.Interpretation of the diverse collection of sensory inputs by thepattern recognizer 70 handles much of the complexity of converting theuser's gestural and touch commands into message meaning data thatcorrelate to functions that the consumer electronic equipment 48 canperform. The resulting division of labor produces a control system thatprovides both a very high, visually engaging information content to theuser 40 regarding his or her control system choices, with an equallyrich collection of gestural and touch commands that the user 40 canemploy to get his or her message across to the control system. Comparethis to the conventional push button remote control that requires onebutton, or a sequence of buttons, to be pressed for each desiredfunction, with the added inconvenience that the user 40 must look at theremote control in order to find the desired button to push.

From the foregoing, it will now be appreciated that the description andbroad teachings of the present disclosure can be implemented in avariety of ways. Therefore, while this disclosure has been describedwith particular examples thereof, the true scope of the invention shouldnot be so limited, as other modifications will become apparent to thoseof skill in the art upon study of the drawings, specification andfollowing claims.

1. A remote control system for consumer electronic equipment having adisplay screen, comprising: a control circuit coupled to said consumerelectronic equipment for controlling operation thereof; a remote controlunit having a handheld case and plurality of sensors disposed withinsaid case, including: (a) at least one touchpad responsive tomanipulation by a user's fingers; and (b) at least one additional sensorselected from the group consisting of: (1) acceleration sensorsresponsive to movement of the remote control unit; (2) case perimetersensors that discriminates which parts of the case are in contact withthe user's body; (3) pressure sensors responsive to pressing forces upona predetermined region of the touchpad; and (4) RF signal sensorsresponsive to radio frequency signals transmitted from said controlcircuit; said remote control unit being configured to produce a sensoryinput message containing information extracted from said plurality ofsensors; a pattern recognizer receptive of said sensory input message;said pattern recognizer being configured to classify the receivedsensory input message according to a predetermined recognition scheme togenerate message meaning data; said pattern recognizer being furtherconfigured to classify the received sensory input message based oncontext information extracted from knowledge of what is being displayedon the display screen of the electronic equipment; control logicresponsive to said message meaning data to generate at least one commandthat is communicated to said control circuit.
 2. The remote controlsystem of claim 1, wherein said at least one additional sensor comprisesat least one acceleration sensor and at least one case perimeter sensor.3. The remote control system of claim 1, wherein said pattern recognizerincludes a trained model operable to classify the received sensory inputmessage.
 4. The remote control system of claim 3, wherein said trainedmodel is adaptable based on interaction with the user.
 5. The remotecontrol system of claim 1, wherein said remote control unit includespower management system and wherein said control logic generates atleast one command that is communicated to said power management systemto alter power management system behavior based on sensory inputmessages said sensory input message.
 6. The remote control system ofclaim 1, wherein said remote control unit includes power managementsystem and wherein said control logic generates at least one commandthat is communicated to said power management system to alter powermanagement system behavior based on said message meaning data.
 7. Theremote control system of claim 1, wherein said control circuit includesmemory for storing control system operation information and wherein saidcontrol circuit selectively provides said operation information to theconsumer electronic equipment for presentation to the user.
 8. Theremote control system of claim 7, wherein said control circuit providessaid control system operation information to the consumer electronicequipment in accordance with said at least one command.
 9. The remotecontrol system of claim 1, wherein said control circuit includes memoryfor storing control system operation information that is integrated withdifferent message meaning data; and wherein said control circuitselectively provides said operation information to the consumerelectronic equipment in accordance with the message meaning datagenerated by said pattern recognizer.
 10. The remote control system ofclaim 1, wherein said at least one additional sensor is an RF signalsensor responsive to radio frequency signals transmitted from saidcontrol circuit and wherein said control circuit is configured to send afind-me message that causes the remote control unit to perform anattention attracting operation useful in assisting the user to find theremote control unit when it has become misplaced.
 11. The remote controlsystem of claim 1, further comprising two touchpads disposed inside-by-side relationship each being separately responsive tomanipulation by the user's fingers.
 12. The remote control system ofclaim 1, wherein said touchpad is divided into plural separate regionsof interaction, such that each region of interaction is interpreted tohave a different meaning by said pattern recognizer.
 13. The remotesystem of claim 1, wherein said at least one additional sensor comprisesa case orientation sensor capable of differentiating between portraitorientation and landscape orientation.
 14. The remote control system ofclaim 1, wherein said touchpad is divided into plural separate regionsof interaction, such that each region of interaction is interpreted tohave a different meaning by said pattern recognizer, wherein said atleast one additional sensor comprises a case orientation sensor capableof differentiating between portrait orientation and landscapeorientation; and wherein said regions of interaction are remapped ontodifferent locations within the touchpad based on case orientation. 15.The remote control system of claim 14, wherein said at least oneadditional sensor further comprises the at least one acceleration sensorand at least one of the case perimeter sensors, said pattern recognizerredefines a zone of interaction, by interpreting how the user is holdingsaid remote control unit based on information extracted from said atleast one additional sensor.
 16. The remote control system of claim 15,wherein said pattern recognizer interprets whether the user is holdingsaid remote control unit with one hand or with two hands.
 17. A remotecontrol system for consumer electronic equipment having a displayscreen, comprising: a control circuit coupled to said consumerelectronic equipment for controlling operation thereof; a remote controlunit having a handheld case and plurality of sensors disposed withinsaid case, including: (a) at least one touchpad responsive tomanipulation by a user's fingers; (b) at least acceleration sensorresponsive to movement of the remote control unit; (c) a case perimetersensor that discriminates which parts of the case are in contact withthe user's body; (d) at least one pressure sensor responsive to radiofrequency signals transmitted from said control circuit; said remotecontrol unit being configured to produce a sensory input messagecontaining information extracted from said plurality of sensors; apattern recognizer receptive of said sensory input message; said patternrecognizer including a trained model adaptable through interaction withthe user and configured to classify the received sensory input messageaccording to a predetermined recognition scheme to generate messagemeaning data; said pattern recognizer being further configured toclassify the received sensory input message based on context informationextracted from knowledge of what is being displayed on the displayscreen of the electronic equipment; control logic responsive to saidmessage meaning data to generate at least one command that iscommunicated to said control circuit; wherein said remote control unitincludes power management system and wherein said control logicgenerates at least one command that is communicated to said powermanagement system to alter power management system behavior based onsaid message meaning data; an wherein said control circuit having memoryfor storing control system operation information and wherein saidcontrol circuit selectively provides said operation information to theconsumer electronic equipment for presentation to the user wherein saidcontrol circuit selectively provides said operation information to theconsumer electronic equipment in accordance with the message meaningdata generated by said pattern recognizer.
 18. The remote control systemof claim 17, wherein said power management system performs: placing ofsaid plurality of sensors included in said remote control unit, into asleep mode; detecting movement of said remote control unit by wakingsaid acceleration sensor up in the sleep mode; determining whether ornot said remote control unit is held by the user, by waking said caseperimeter sensor up when the movement of said remote control unit isdetected; and releasing the sleep mode when the detecting detects thatsaid remote control unit is held by the user.
 19. The remote controlsystem of claim 18, wherein said power management system releases thesleep mode, by waking said at least one touchpad up and then waking saidRF signal sensor up, when the detecting detects that said remote controlunit is held by the user.
 20. A method of controlling consumerelectronic equipment comprising; providing a remote control unit havingat least one touchpad; sensing manipulation by a user of said at leastone touchpad; sensing physical movement of said remote control unit;classifying said sensed manipulation and said sensed physical movementby pattern recognition; sending at least one control command to theconsumer electronic equipment based on the results of said classifyingstep.
 21. A remote control system for consumer electronic equipmenthaving a display screen, comprising: a control circuit coupled to saidconsumer electronic equipment for controlling operation thereof; aremote control unit having a handheld case and plurality of sensorsdisposed within said case, including: (a) at least one touchpadresponsive to manipulation by a user's fingers; and (b) at least oneadditional sensor selected from the group consisting of: (1)acceleration sensors responsive to movement of the remote control unit;(2) case perimeter sensors that discriminates which parts of the caseare in contact with the user's body; said remote control unit beingconfigured to produce a sensory input message containing informationextracted from said plurality of sensors; a pattern recognizer receptiveof said sensory input message; said pattern recognizer being configuredto classify the received sensory input message according to apredetermined recognition scheme to generate message meaning data;control logic responsive to said message meaning data to generate atleast one command that is communicated to said control circuit, whereinsaid at least one additional sensor further comprises the at least oneacceleration sensor and at least one of the case perimeter sensors, saidpattern recognizer redefines a zone of interaction, by interpreting howthe user is holding said remote control unit based on informationextracted from said at least one additional sensor; wherein said remotecontrol unit includes power management system and wherein said controllogic generates at least one command that is communicated to said powermanagement system to alter power management system behavior on saidmessage meaning date; and wherein said power management system performs:placing of said plurality of sensors included in said remote controlunit, into a sleep mode; detecting movement of said remote control unitby waking said acceleration sensor up in the sleep mode; determiningwhether or not said remote control unit is held by the user, by wakingsaid case perimeter sensor up when the movement of said remote controlunit is detected; and releasing the sleep mode when the detectingdetects that said remote control unit is held by the user.
 22. Theremote control system of claim 21, further comprising: (3) pressuresensors responsive to pressing forces upon a predetermined region of thetouchpad; and (4) RF signal sensors responsive to radio frequencysignals transmitted from said control circuit; said pattern recognizerbeing further configured to classify the received sensory input messagebased on context information extracted from knowledge of what is beingdisplayed on the display screen of the electronic equipment.